In various multi-tenant environments (e.g., cloud environments, datacenters, etc.), several host machines operate to host virtual machines (VMs) for the different tenants of the multi-tenant environment. In some cases, several workload (or guest) VMs of various different tenants can operate on a single host, maintaining a logical separation so that traffic for the workload VMs of the different tenants is isolated from each other.
Increasingly, in such shared environments, security services (as well as other services) must be applied within the datacenter, not only against external threats, but as well as from threats of other machines within the datacenter. In some such cases, the services are distributed and enforced throughout the network. For example, a distributed firewall provides firewall services with multiple enforcement points throughout the network (e.g., at hypervisors that operates on each host machine).
However, distributing the services comes with a cost, as the services consume resources in the host to be used by the VMs. This is an important factor to consider when deciding the number of workload VMs that can be effectively run on the host. There is no way for the administrator to identify how much resources are being consumed by different distributed services (e.g., firewall, load balancing, anti-virus, etc.). As the services are distributed throughout the network, they must also be scaled as new host machines (for new workload machines) are added to the network.
While resources for the workload VMs are often managed by virtualization layers that operate on the hosts, services provided for the workload VMs by each host are not similarly managed. It is often possible for network traffic from certain VMs to use a majority of the service resources (e.g., processing and memory resources dedicated to providing the services), starving the other VMs that share the service resources on the host. For example, VMs for a particular tenant could create a huge number of connections that fill up a heap for a distributed firewall (DFW), monopolizing the service resources and preventing VMs of other tenants from creating new connections. Similar problems can also arise between different services, different providers of the services, etc.